Papers with autoregressive baselines
Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generation (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing non-AutoRegressive (NAR) text generation models lack proper pre-training, making them far behind pre-trained autoregressive models. |
| Approach: | They propose a novel pre-training task to promote prediction consistency in non-autoregressive (NAR) generation. |
| Outcome: | The proposed model outperforms existing pre-trained models and achieves 17 times speedup in throughput. |
Text Editing as Imitation Game (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Text editing is an important domain of processing tasks to edit the text in a localized fashion, such as text simplification. |
| Approach: | They propose a nonautoregressive decoder for state-to-action demonstrations that parallels the decoding while retaining the dependencies between tokens. |
| Outcome: | The proposed model outperforms the autoregressive baselines on a suite of Arithmetic Equation benchmarks in terms of performance, efficiency, and robustness. |
TAPE: Assessing Few-shot Russian Language Understanding (2022.findings-emnlp)
Copied to clipboard
Ekaterina Taktasheva, Tatiana Shavrina, Alena Fenogenova, Denis Shevelev, Nadezhda Katricheva, Maria Tikhonova, Albina Akhmetgareeva, Oleg Zinkevich, Anastasiia Bashmakova, Svetlana Iordanskaia, Alena Spiridonova, Valentina Kurenshchikova, Ekaterina Artemova, Vladislav Mikhailov
| Challenge: | Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes, but lacks standardized evaluation suites for non-English languages. |
| Approach: | They propose a novel benchmark that includes six more complex NLU tasks for Russian, covering multi-hop reasoning, ethical concepts, logic and commonsense knowledge. |
| Outcome: | The proposed benchmark includes six more complex NLU tasks for Russian, covering multi-hop reasoning, ethical concepts, logic and commonsense knowledge. |
Explicit Syntactic Guidance for Neural Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing text generation models follow the sequence-to-sequence paradigm . generative grammar suggests humans generate language by learning language grammar . |
| Approach: | They propose a syntax-guided generation schema that searches the syntax tree in a top-down direction. |
| Outcome: | The proposed method outperforms autoregressive baselines on paraphrase generation and machine translation. |